The present invention relates generally to the field of data processing, and more particularly to data mapping and reducing systems and methods.
A map and reduce program is composed of a mapping operation that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a reducing operation that performs a summary function (such as counting the number of students in each queue, yielding name frequencies). A mapping and reducing system (also called “infrastructure” or “framework”) orchestrates the processing by marshalling the distributed servers, running the various tasks in parallel, managing all communications and data transfers between the various parts of the system, and providing for redundancy and fault tolerance.